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PREFACE 



The Research on Evaluation P ogram is a Northwest Regional 
Educational Laboratory project of research, development, testing, 
and training designed to create new evaluation methodologies for 
use in education. This document is one of a series of papers and 
reports produced by program staff , visiting scholar adjunct 
scholars, and project collaborators—all members of a cooperative 
network of colleagues working on the development of new 
methodologies* 

How does one collect photographic information for use in 
evaluations? what are the major technical and procedural issues 
of concern in doing such studies? This report provides 
assistance in designing, conducting, and reporting evaluations 
using a visual documentary approach, featuring the use of still 
photography. It covers a wide range of both practical and 
methodological concerns in using photography in evaluation. 



Nick L« Smith 

Editor, Paper and Report Series 
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Introduction 



This handbook is intended to help educational evaluators use 
still photography in designing, conducting, and reporting 
evaluations of educational programs. It describes techniques for 
using a visual documentary approach to program evaluation that 
features data collected with a camera. In this handbook, 
photography refers to still photography, particularly black and 
white photographs. 

Faced with ever-increasing pluralism, with the current 
requirements for accountability, and the problems of limited 
resources—decreasing revenues, declining enrollments, shrinking 
programs, aging teachers, and more heterogenous student 
bodies—evaluators are assuming more responsiblity for using 
descriptive and qualitative methods of doing evaluation, rather 
than strict treatment and experimental control designs that seek 
specific causes and effects. More and more, the central 
questions asked are about how programs operate and what their 
purpose is, rather than what their outcomes are. Questions like 
these are asked: ^ 

What is the current state of affairs? 

What does the program look like to different people? 

What is the shared educational experience? 

What is the daily experience of students and teachers? 

What goes on here? 

This shift in questions is accompanied by a shift in methods 
of approaching evaluatic s— from causal to non-causal models. I 
am referring to the inc ase in the use of multiple methods, case 
studies, photographic documentaries, and other descriptive 
approaches. The plentiful supply of models given to qualitative 
methods attests to the interest and willingness among evaluators 
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to consider new methods for solving evaluation problems. Various 
qualitative approaches are associated with responsive evaluations 
(Stake, 1978a): case study (Stake, 1978a, 1978b), noncausal 
inquiry (Smith, 1981), field work (Burnett, 1973; Smith, 1978), 
naturalistic inquiry (Guba and Lincoln, 1980), educational 
connoursiership {Eisner, 1979), and thick description (Geertz, 
1973). The general trend integrates multiple methods of 
observation and interviewing for collecting data, and plausible 
inference and logical analysis for interpreting data. 

In addition to the shift in era{Jhasis from causal to noncausal 
models, more attention is being given now to the iiqpact of 
evaluation results on policy decisions,, and their influence on 
improving instructional programs. A chief concern is with not 
only the quality of evaluation evidence but also effective ways 
of presenting reports to decision makers. Documentary 
photography is one of the variety of alternative qualitative 
reporting styles being considered. Visual information used by 
evaluators provides one form of program knowledge in response to 
these practical concerns about the quality of methods and 
evidence. 

Evaluators generally have had limited experience in using 
photography for purposes of program evaluation. Few have been 
trained in its use* Some evaluations have been made in which 
photographs are a prominent form of evidence; they are cited 
throughout this book. Many disciplines have used photography in 
research and evaluation: archeology, medicine, physiology, 
zoology, botany, and psychology, to name a few. The major 
contributions to the methodology of documentary social inquiry 
have come mainly from photo- journalists, sociologists, and 
ethnographers* These people have developed a variety of 
systematic methods for using a camera as a research and 
evaluation tool. The approach adapted here for program 
evaluation cones mainly from the traditions of visual 
anthropology, visual sociology, and documentary photography, 
utilizing methods of participant observation. 



3 

2 



What are the practical considerations involved in developing 
and carrying out an evaluation using photography? A sound method 
of photographic inquiry contains at least ten elements, all of 
which will be described in this handbook: 

• Deciding when to use visual information 

• Planning photographic techniques 

• Selecting the photographer (s) 

• Identifying program issues 

• Selecting a sampling method 

• Dealing with reactivity 

• Collecting the data 

• Establishing validity 

• Interpreting the data 

• Reporting evaluations using photography 

Deciding when to Use Visual Information 

Photography is about communication. Photographs may tell a 
story or illustrate one. Evaluators may use series of 
photographs to communicate a story in an appraisal of a program 
or to reinforce valuations by illustrating (House, 1980). 
Evaluators are choosing to use photography i n a variety of 
situations. Following is a listing of conditions when evaluators 
might choose to use photography for 

• a record of events in detail 

• close-up work with program participants 

• visual information that is of primary interest 

• tracking the activities of a single participant in 
the program 

• thick description of the process in a program rather 
than outcomes 
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• information £bout how the program looks different to 
various people 

• documentation of the context of the program 

• a record of information about social interaction of 
particular individuals, groups, or communities 

• evaluating the daily conditions in the program 

• documenting regular routines of regular and special 
classes or programs 

• descriptions of particular learning media 

• answers to a variety of questions that can be asked 

• an interdisciplinary effort that is developing,* each 
asking different questions 

• plausible interpretation^ rather than explanations 

• ^ designs that are flexible 

• activities rather than goals of the program 

• understanding the subjective nature of the 
participants* experience 

• discovering the unexpectd, unobtrusive, secondary 
effects of the program 



Introduction 

This section contains a discussion of various technical 
aspects of photography in evaluation. The procedures used in 
several studies are described, as are the reasons for choosing 
them* Alternatives that might work as well as the most desirable 
procedures do are included. Also included in a description of 
the qualifications the photpgrapher(s) should have. 



* 

I wish to express thinks to Michael Sawdey and Adrianne 
Bank for their contributions to this section. 



loolcingNtt a set of social relationships 




Planning Photographic Techniques * 
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Equipment 

The vast majority of photography in evaluation is done 
rapidly, unobtrusively, and in available light. It involves 
taking a large number of pictures • The conditions more or less 
require that the photographs be taken with 35mm equipment, most 
often with the ubiquitous single-lens reflex, camera. Whatever 
camera is used, it will need to have a lens with an aperture of 
at least f .2.8. to permit available light photography in average 
office and classroom lighting. 

In the evaluation of the Evaluation Network Conference 
(Templin, 1979), the two photographers used 35mm single-lens 
reflex cameras (Miranda G and Olympus OM-1), taking most of the 
pictures with the "normal" 50mm lenses of these camera. A few 
shots (2-3% of the total) were taken with an 85mm "short 
telephoto" lens. This made it possible to examine details at a 
greater distance, but was certainly not essential • In that 
study, a wide-angle lens would have been useful in some 
situations. For example, as the conference progressed/ one 
feat tire of the meetings that we sought to examine 
photographically was the buffer zone that develops between the 
presenter and the audience. Because of the limited space in the 
rooms, it was often difficult to include enough angle of view to 
show the extent and configuration of the buffer zone. A 
wide-angle lens would have helped. Again, this was not 
essential: by taking two or three successive shots from one 
vantage point and piecing the resulting prints together 
edge-to-edge, it was possible to provide a panorama of a room. 
Although this technique is less elegant than using a wide-angle 
lans, it has the advantage that it does not introduce distortion 
of distance or perspective. In this connection it must be 
remembered that wide angle lenses tend to exaggerate the apparent 
distance between objects arranged at different distances from 'the 
camera, whereas a telephoto lens tends to compress this 
distance., If one wanted to interchange the normal lens for 
telephoto, wide-angle, or close-up lenses, the single-lens reflex 
camera accommodates the other lenses. 
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The single-lens reflexes did- not present any severe 
difficulties in collecting the data '-at the Evaluation Network 
Conference, although two considerations might be important under 
some other circumstances. First/ this type of camera is 
relatively noisy;- and thus tends to be obtrusive. Second/ in 
e*tr*oely poor light the single-lens reflex can be difficult to 
focus. However, the lighting in the conference roods was 
adequate for focusing. To overcome the problems of noise and the 
uncertainties of focusing in low light, some photographers prefer 
to use 35mm rangefinder cameras. Most of the moderately priced 
rangefinder cameras now available do not take interchangeable 
lenses, and their fixed lenses' genera Hy have a siightly wider 
than normal aitgle of view. For most purposes these would not be 
significant drawbacks f and the use of compact 35mm rangefinder 
cameras for evaluation photography should be explored. 

Photographic Materials and Processing 

* « 
It is not usably necessary to use ultra-high-speed films for 

evaluation photography, A film with an ASA speed of 400 (such as 

Kodak Tri-X 6r Ilf ord HP-4) is sufficiently fast for photography 

- in ordinary room light, but will still provide fine-grained 

photos that wiU enlarge well. Likewise, there is no need to use 

exotic developers in the darkroom. Standard developers (such as 

Kodak D-76 or Agfa Rodinol) are inexpensive and provide plenty of 

ape*d and contrast for the purposes of evaluation photography. 

At the Evaluation Network Conference, the photographs were 

taken with Kodak Tri-X film, using exposures determined for the 

normal speed of this film (ASA 400). The available light in the 

conference rooms required exposures ranging from 1/30 second at 

12 in the dark corners to 1/60 second at f4 m the brightest 

areas. Most classrooms and office areas would probably be 

somewhat brighter than the brightest parts of the conference 

rooms, so it *ould seem that availability of light wouh^e^ 

normally be a problem in evaluation photography. 

In evaluation photography it is usually necessary to develop 

and print pictures daily, as detailed later m this section. 
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This is not as difficult as it sounds, even if there is no 
darkroom immediately available. Film can be loaded into 
developing tanks in room light by using a changing bag. A 
darkroom for printing can be set up in almost any bathroom that 
can be more or less darkened. A compact en larger can be carried 
in a suitcase, plastic chemical trays Uned up in the bathtub, 
and prints made on resin coated paper that requires no special 
equipment for drying. 

The photographs taken at the Evaluation Network Conference 
were developed in this sort of temporary darkroom. Film was 
developed in Kodak D-76 developer, allowing development times 
about 10% longer than those recommended by the manufacturer. 
This provides a slight increase in contrast and helps show more 
detail in the shadows; both of these factors make it easier to 
examine fine details in the completed photographs. "Push 
processing" to increase the light sensitivity of the film was not 
used becsuse there was adequate light to expose the film 
normally, and because pushing film increases the graininess of 
the resulting photos— again, obscuring detail. Other aspects of 
the film processing were normal, except that the film was 
occasionally dried with a blow-type hairdryer to speed processing 
time. In cases where film must be processed very rapidly, the 
following steps can be taken.- 1) use rapid fixer; 2) soak film 
xn hypo eliminator to reduce washing time; ^3) dry film instantly 
by dipping in film drying solution (a volatile solvent that 
displaces the water from the surface of the film). 

All negatives were contact-printed on 8x10 inch photographic 
paper and,, after selection of frames to be enlarged, 8x10 
enlargements were made. Because of methodological 
cone iderat ions, the entire frame was always enlarged. This 
results in a print with wide borders on its long sides, since the 
proportions of the 35mm frame (24 x 36mm) are different from 
those of the standard 8x10 photographic paper. All prints were 
made on resin-coated paper (in this cas%, from liford, although 
similar papers are made by Kodak, Agfa, and ail other major 
manufacturers). Resin-coated papers do not absorb water; thus 



they process and wash more quickly than conventional papers and 
they can be dried without special equipment by simply hanging 
them up with clothespins. For processing in the field, these 
advantages outweigh the fact that resin-coated papers are 
somewhat more expensive, have slightly less range of gray tones, 
and are less stable in archival storage than conventional 
photographic papers. Glossy paper was used because the surface 
texture of matte papers may sometimes obscure detail in a 
photograph. 

Many of the photographs taken at the conference were also 
made into black-and-white slides. This is a very simple 
process: the negatives are contact printed on positive print 
film in the darkroom. This Ko^ak product is a very slow (ASA 4) 
35mm film which can be handled in the darkroom under a red 
safe-light and developed in the developer normally used for 
photographic prints (e.g., Dektol). The positive print film was 
cut into strips a little longer than the strips of negatives and 
then the negatives and print film were sandwiched together under 
a piece of glass and exposed briefly under the enlarger. With 
practice, one can perform this process rapidly and readily 
produce a large quantity of slides for projection and examination. 

After the strips of film were fixed, washed, and dried, they 
were cut apart into individual frames and mounted in Pako 
one-piece plastic slide mounts* These mounts are considerably 
easier to use than two-piece plastic mounts, or cardboard mounts 
that must be heat-sealed. 

All darkroom work was successfully performed in a motel 
bathroom, indicating that there should be few proLloms in 
processing black-and-white film and making prints and slides "on 
location". A changing bag was used for loading film into the 
processing tank. This procedure is recommended because it may be 
difficult to make a room dark enough for film handling, although 
the motel bathroom was sufficiently light tight for making prints 
and slides. When working under time constraints, the major 
processing bottleneck proved to be the 4-roll film tank. It is 
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recommended that a larger tank (e.g., 8- or 10-roll capacity) be 
used to 3ped processing. 



Record Keeping 

If still photography is to be used seriously for evaluation 
purposes, it is essential that complete records be kept during 
data collection. During the Evaluation Network Conference, the 
photographers used a newsprint pad and felt-tip pen to record the 
date, time, location, and subject matter, together with a roll 
number assigned to each roll of film. This newsprint page was 
then photographed at the beginning of the roll, and at every 
change of subject matter or location a new sheet was written up 
and photographed* 

After processing, ?5mm negatives are commonly cut into strips 
of six exposures each, since this length fits conveniently onto 
an 8x10 sheet of photo paper for contact printing. The strips 
from each roll were placed in a film storage envelope and 
identified with the nmber of the roll, taken from the newsprint 
sheets photographed on the roll. 

Each roll of 35nun film is marked at the factory with frame 
numbers along its edge. Thus, when the film from a roll is 
contact printei, it is possible to arrange the strips of 
negatives i^n the order in which they were exposed. Before 
processing the contact sheet, the roll number was written on the 
back. In this *ay, each exposure on the sheet can be identified 
for future reference by referring to the roll number and frame 
number. We chose to represent this number as roll/frame, 
separating the numbers with a slash. Thus 9/32 - roll 9, frame 
32. 

When enlargements *re made from the negatives, the roll/frame 
number is written on the back of the print. This makes it 
possible to return to "he contact sheet, find the frame, find out 
what frames preceded and followed it, retrieve the information on 
date, time, place, and subject matter from the photo of the 
newsprint sheet, etc. Likewise, when a slide is mounted, the 
roll/frame number is written on the slide mount. 
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If the photographer or another observer is making field notes 
while photos are being taken, then the roll numbers should be 
entered in the notes. It may also be desirable to make notes on, 
say, the number of frames devoted to this or that item covered in 
the notes. This information can be expanded and elaborated later 
by going over the field notes while referring to the contact 
sheets of the photographs. 

Work Output and Costs 

In the Evaluation Network Conference evaluation the two 
photographers exposed a total of 19 rolls of film, or 
approximately 700 exposures, although the last three roils were 
not developed until after the conference due to time 
constraints. Prom the contact sheets, about 200 photographs were 
selected for further study. Of these, about 50 were made into 
8x10 enlargements and approximately 100 were made into 
black-and-white slides. 

Total time spent in photographing during the first two d*ys 
of the three-day conference was approximately 28 person-hours. 
Darkroom work required about 24 person- hours, and related 
operations, such as sorting and selection, record keeping, slide 
mounting, sequencing of slides and prints, etc., required another 
10 person-hour 8. 

Total cost of materials used in photographing the session, 
including film, chemicals, printing paper, slide film, and slide 
mounts, was approximately $55. However , this figure may 
eventually rise to about $75 because more prints and slides will 
be produced later for additional analyses. 

By contrast, in an evaluation of computer literacy in 
non-school settings the costs were considerably higher. The 
commissioned photographer at $2 5 per hour and commercial 
laboratory processing of film and prints totaled approximately 
$80 0. 
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Selecting the Photographer(s) 



Evaluators may wish to serve as photographers in their own ' 
studies. In some instances, it may make more sense to hire, and 
possibly retain, one or more photographers. Should evaluators 
wish to employ photographers to work with them, here are a few 
suggestions selecting a qualified person. 

Questions for interviewing prospective photographers could be 
based on the following considerations: 

1. Visual sense 

2. Technical competency 

3. Philosophy about photography 

4. Social competence 

Visual sense. The photographer must have an aesthetic eye; 
that is, be able to frame a picture in both time and space. This 
ability to know what to photograph and when to close the shutter 
comes partly from training and partly from experience, but it 
does not come to every photographer. A competent, experienced 
photographer will very likely present a collection of pictures in 
a portfolio for the evaluator to examine. 

Technical competence . Technical competence refers generally 
to the ability to set and focus the camera consistently and 
well. This skill is essential. Competence in darkroom 
techniques is also imperative unless the budget includes 
sufficient funds to have film and prints processed by a 
commercial lab. Of particular importance in the darkroom. are 
speed, high standards of quality in developing and printing film, 
and efficiency. In evaluation work, there frequently is a large 
volune of film to handle in a short time. Humor under tension is 
related closely to technical competence because patience is 
required in producing work of high quality while collaborating 
with another person. A good balance of technical competence and 
aesthetic skill is highly desirable. 
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Philosophy about photography . The photographer should be 
experienced in documentary work* Social documentary photography 
requires the photographer to understand that collections of 
photographs rather than single pictures make the kinds of social 
statements appropriate for evaluation purposes* The viewpoint 
that a single photograph is a work of art is more appropriate 
when photos are used for illustration than when they are used as 
data. Having a documentary philosophy of inquiry is vital, 
coupled with a keen ability to observe, be analytical, and make 
connections between the factors in a social situation. Looking 
upon the camera as a research tool also implies the photographer 
knows something about the problems and methods of research and 
evaluation. This is especially important because in many 
evaluations the photographer will have to work fast. 

Social competence . It is necessary in documentary work to 
interact with the participants in a program. Social skill under 
these circumstances determines to some extent the amount of trust 
that will be established between the photographer and the 
participants. A photographer with social intelligence is able 
not only to gather data of higher quality than a photographer 
> without it, but also to relate to people with reassurance, 

empathy, and humor while still maintaining the critical 
perspective and objectivity required for documentary work. 



Identifying Program Issues 

The term "foreshadowed problems" refers to generally defined 
topics of interest that help anticipate' what data are to be 
collected in the beginning of a study (Malinowski, 1922). They 
help focus the topic, organize the evaluation process, anticipate 
the important questions and issues that emerge, arid guide the 

v * 

r* interpretation of the photographs. Using foreshadowed problems 

is a way of discovering what is occurring in a program so that 
the photographer does not go in unprepared. Foreshadowed 
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problems may come from several sources — the field, the 
literature, or theoretical knowledge. 

When an evaluation plan is formulated for a study using 
photography, the evaluator selects some systematic approach to 
identifying what to study and photograph about the program. In 
particular, two approaches seem relevant to documentary 
photographic evaluation: one uses program issues, and the other 
uses telling theories. Program issues and telling theories 
function similarly: they provide a tentative understanding of 
what to expect by offering a viewpoint to either adopt or rule 
out in the course of the study, and they also suggest some 
specific events to photograph. 

Identifying Issues 

Program issues are those few questions used to structure the 
study of a particlar program; they are asked in the context of 
the specific program (Stake, 1980). They are related closely 
enough to the aims and concerns of the program constituents to 
provide vital understanding of the program. 

On the other hand, telling theories are simple concepts or a 
set of concepts that are selected for their relevance to areas of 
concern about the program. They help identify what to photograph 
in order to develop an understanding of the program. They are 
useful for getting ready for the evaluation by anticipating some 
explicit problems and issues that may arise in fully describing 
and making judgments about the mechanics and operations of a 
program. Telling theories help direct evaluators to the issues 
in a program by drawing attention to the most important 
questions, the ones to which the evaluators should direct their 
efforts to find answers (Stake, 1978a). 

Program issues aryd telling theories are held lightly: the 
photographer has to be willing to give them up or change them the 
moment that photographic information reveals something new that 
does not fit the interpretation. Otherwise, one may wind up with 
a large set of disorganized pictures that are hard to make sense 
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of. Worse yet, holding rigidly to prior issues and theoretical 
notions may lead the photographer to excessive coverage of items 
that turn out to be unimportant in the situation, at the expense 
of adequately reporting items that are important. 



Photographers have to know roughly what to take pictures of 
before they begin, and what to photograph next. T^heTe are too 
many things to photograph to be able to enter the evaluation 
situation without sane preconceptions. When taking pictures for 
evaluation purposes, there may be a tendency in the beginning to 
photograph everything, including non-participants. This seems to 
be inevitable, and, in the interest of providing seme margin for 
error, is not usually a cause for concern. Some things go by 
fast, take place simultaneously, or will never happen again. In 
other situations some things will be mere stable, in any case, 
program issues and telling theories provide the initial guidance 
needed to begin taking pictures, without blinding the 
photographer to the unanticipated elements that emerge as the 
photographing proceeds. 

The use of the term theory may conjure up suggestions of 
doing research rather than evaluation. The term is used here 
instead only to acknowledge the origins of the mental activity, 
interpretive in nature, that often lead evaluators 'to observe the 
kinds of events they do in their attempts to improve 
understandings of the program. These abstract contributions are 
likely to transform into more concrete issues during the course 
of the evaluation. 

The selection of issues or theories for photo-assisted 
evaluation is not markedly different from that of studies 
utilizing other data collection techniques. It is necessary, 
however, that the questions raised by the issues or telling 
theory be ones that can be satisfactorily answered by 
photographic data. Such questions usually involve the 
interaction of persons with one another and with the environment, 
in both space and time. 



Selecting Telling Theories 
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Initially, the selection of telling theories arises from two 
types of information: existing data about the situation to be 
evaluated, and existing literature that provides more abstract 
ideas of potential descriptive utility. The selection of issues 
arises from data about the programs operation or that of similar 
educational programs. Existing information about the situation 
(outreach literature, program curricula, instructional plans, 
floor plans, activity schedules, etc.) can provide a sense of 
what the planners, administrator*, teachers see in the situation, 
and what they expect to happen. Likewise, existing theoretical 
ideas can provide, an initial sense of what is likely to happen in 
the situation, and what events may have significance. The 
questions one then asks—and attempts to answer with photographic 
evidence— guide attempts to either confirm or disconf irm aspects 
of the anticipated structure of the situation. First in 
tentative form, and then in modified form, both program issues 
and telling theories help shape the procedures for sampling, 
collecting data, and interpreting the results. 



Selecting a Sampling Method 

Traditional causal research methods assume that when one 
samples, one is drawing randomly from some population of possible 
cases. Deviation from the true nature of the defined population 
is referred to as sampling error, which limits the 
generalizability of findings to others in the population. In 
non-causal evaluations that use photography in describing and 
appraising programs, a different concept of sampling is used. 
Here, sampling, emphasizes how representative the sample is of the 
population, and of the social and physical environment in which 
the program occurs. Generalizability to other programs is 
assumed to be more a matter of illumination, based on the 
readers experience with and knowladge about similar programs 
(Stake, 1980; Parlett and Hamilton, 1977). 
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Sampling decisions are made to obtain photographs that fairly 
represent the events and participants in a program. One key idea 
is to represent the multiple perspectives of the various 
constituents in a program in both time and space. 

Evaluators using photography attempt to answer three general 
sampling questions: 

1. Is this photographic evidence representative? 

2. Of what is it representative? 

3. Whose differing perspectives are represented? 

4. Does this sample of photographs represent to others 
the interpretation the evaluator intended? 

While composition may imply what is important in a single 
picture/ sampling implies what is important in a collection of 
documentary photographs. Collier (1967) claimed that sampling in 
photography connotes authenticity. The model of sampling applied 
in this photographic method is Glaser and Strauss' (1967) notion 
of theoretical sampling for purposes of generating explanations. 
Units of observation (categories of information or issues) are 
chosen because some emerging issues suggest they are strategic 
for understanding the data. 

Social scientist^ in general deal with threats to the 
validity of assertions and inferences by applying various 
conventional sampling techniques that allow certain kinds of 
generalizations or plausible inferences to be made. 
Photographers may want to show their photographs in a way that 
implies generalizability, that what they say applies to a wider 
population. Some sampling techniques pertinent to photography 
include time sampling, checking up on people at regular intervals 
(minutes, hours, days, weeks, seasons), or sampling at intervals 
to get a full range of times and activities* Another sampling 
technique entails shadowing or following one person through an 
entire daily or other routine. A third approach is snowball 
sampling, asking each subject who else the investigator should 
talk to, and what else should be investigated, and photographed, 
letting one subject lead to another. 
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Using issues as a sampling device may direct the 
photographer's attention to things toward kihich the biases of 
interest or intuition might not otherwise lead them, things that 
indicate what might constitute a fuller description of the 
situation under 3tudy. Photographing certain activities or 
places following a plan counteracts the tendency to photograph 
only what appears personally interesting. Photographers may 
expose a roll of film every hour, for example. Techniques of 
randomization and theoretically informed sampling also might 
direct the photographer to use a different technique at some 
point to gather evidence—shooting from low, high, or varied 
angles, from behind the subjects, over the shoulder, from the 
torso with preset exposure and distance. Whatever sampling 
devices are used, photographers need to identify the photographs 
by date, time, film roll and frame, and place, in addition to 
describing the overall sampling strategy. 

Time- Baaed Sampling 

In this type of sampling, the viewpoint and spatial coverage 
of the camera are fixed and photographs are taken at regular time 
intervals, sometimes ranging from one frame per second to one 
frame p*r minute. One method of collecting data with this method 
is to use a 16mm movie camera fitted with an extreme wide-angle 
lens (often covering a whole room), and an intervalometer. 
Alternatively, a 35mm still camera can be used, fitted with a 
bulk film magazine, motor drive, extreme wide-angle lens, and 
intervalometer. The advantage of the movie camera is that the 
resulting film can be projected as a time-lapse movie, thus 
showing motion patterns and changes in the spatial distribution 
of persons in the environment. On the other hand, time-based 
photo sequences taken with the 35mm still camera can be examined 
minutelv for such details as eye contact, and prints can be 
measured to quantify the proximity of subjects to one another. 

An advantage of time-based sampling is that it accurately 
reveals shifting patterns of persons in space 6v* r time. As 
such, it can be extremely useful in studies of the use of space 
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and also can provide information on the changing relations 
between groups and between members of groups. A disadvantage of 
this method is suggested by Goff man's (197 6) observation that 
human interaction is largely ceremonial, being concentrated in 
liminal events: collecting data on a constant time base may tend 
to exaggerate the importance of the large proportion of 
"inactive"- time that tends to separate such liminal events. 

Shadow Sampling 

This method involves following a single subject through the 
program, recording the experiences and interactions of that 
individual. An advantage of this method is that the photographic 
data collected will all share a common independent variable: the 
individual being shadowed. This is of course also a disadvantage 
in that the individual chosen for shadowing may introduce 
systematic bias into the data through the types of interactions 
and situations in which she or he typically and habitually 
engages (or does not engage). Taking this viewpoint, shadow 
sampling may result in collecting data about the individual, 
rather than about the program. This problem can be overcome to 
some extent by shadowing several subjects, either simultaneously 
or consecutively. However, simultaneous shadowing generally 
requires multiple photographers, which is likely to be very 
obtrusive, while consecutively shadowing several subjects through 
a program of any length may require \unrealistic amounts of 
contact time. The payoff is description of a participant's 
experience. 



Blanket Sampling 

This method does not constrain the photographer to follow a 
given session or event through to completion. Rather, the 
photographer attempts to cover as much of the entire span of 
events and sessions as possible by moving about freely, and 
fairly constantly, from location to location. When this method 
is followed, it seems likely that the photographer will gather a 
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rather large proportion of lirainal events, generally at the 
expense of sequences showing development of action, changes of 
groups, shifts in proxemics, etc. Basically, this is the 
methodology of traditional photojournalism, and the fact that it 
tends to gather only liroinal events probably accounts for both 
the strong impact and the uncontrolled biases of most 
photographic documentaries. 

Event-Based Sampling 

Sometimes the research question posed for a study is not 
concerned with the program or system as a whole, but with a 
particular type of action, interaction, sequence, relationship, 
or event within the overall context. In such cases, the 
photographer may be assigned to sample as many events as possible 
within a certain category. Such categories may be relatively 
broad ( "question-and-answer periods," "peer teaching at 
terminals"), or specific ("handshakes between persons of the 
opposite sex after the presentation"). Whatever the category, 
•vent-based sampling is generally appropriate in response to 
specific questions or program issues that are emerging. It 
should probably be accompanied by some more global technique 
(e.g., blanketing the event area, having another person ta) 
field notes, tape recording) in order to provide contextual 
information for interpreting the category of specific events. 

Dlmenaionally-Based Sampling 

The development of a dimensionally based sampling procedure 
for the EN Conference evaluation is described elsewhere. 
However, the basic principles of this type of sampling will be 
outlined here for ease of application in other circumstances, 
Dimensionally based sampling would appear to be useful where the 
initial aim of the study is descriptive appraisal, but the 
constraints of time, space, and work force make it impossible to 
cover all events in the program. Any information available about 
the program (field notes, previous reports, informants* reports, 
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printed programs, informational brochures, etc*) is^fcammed to 
determine a set of apparent dimensions for classifying the events 
and components of the program* Events are then selected for 
-photographing to represent the largest possible range of 
dimensions and combinations of dimensions. Other constraints, 
such as photographing a whole event, or spending a predetermined 
amount of time at each event, may also be imposed. 

The advantage of this sampling procedure is that it offers 
some guarantee of representativeness in the description of the 
program. It should be noted, however, that, in a photographic 
study of any length, this method will usually need to be 
progressively modified during the course of the study to reflect 
the specific program questions that arise as a result of analysis 
of the photographic data. Once it has been established that the 
data collected are truly representative of the program, then it 
may be appropriate to initiate more rpecific event-based sampling 
to study apparently significant or pivotal aspects of th* program 
in greater depth* 
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Dealing with Reactivity 

The traditional concept of reactivity refers to the influence 
of the observer's presence on the behavior of participants in the 
program. It raises the question of whether the events observed ^ 
and recorded by che photographer are an accurate reflection of * 
ordinary behavior or a result of the photographer's presence. 

In social documentary inquiry, the camera is considered by 
Becker (1974, 1979) and Collier (1967) to help reduce ' 
reactivity. They argue that carrying a camera helps ;,alidat.i the 
investigator* s right to be there. People generally want their 
picturea taken, whoever the photographer is. Under many 
circumstances, being obe ved and photographed is commonplace, 
expects, and accepted. At least people know specifically what 
kind of data an observer is collecting with a camera, even though 
the presence of the canera may produce sane a:ixiety at first. 
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The camera is an obtrusive instrument for gathering 
evaluation data. The camera is not easily ignored, and, even if 
they do not do so deliberately, people do react to its presence. 
This may be true of observers in general, of course, and the fact 
that reactivity occurs in response to a photographer is not a 
very unusual or difficult problem. The camera may actually be 
part of a strategy for the evaluator entering the situation. It 
may help an evaluator gain access to the situaeion because the 
presence of a person uai. j a camera is largely self-explanatory. 
Yet, the camera forces immediate explanation and interaction 
between the photographer and the participants, in general, a • 
person photographing for an evaluation study should have a 
prepared explanation for his or her presence: it should be brief 
and matter of fact, much like the sort of prepared statement a 
survey interviewer uses as a preface to an interview. 

Becker (1974, 1979), Collier (1967), and Walker ary/wiedel 
(1979) claimed that the camera facilitates the photographer's 
access and helps in developing trusting relationships between the 
participants and the photographer. The kind of relationships 
developed are closely related to how revealing the data can be 
and to the kind of interpretations the- are possible in a study. 
This is particularly true in photo-interviewing when photographs 
are shown back to subjects for purposes of interviewing them 
ebout. what has been depicted, when and where the photographer is 
able to put himself or herself in relationship to the subjects 
indicates the» trust that has been established. The social and 
physical distance between participants and photographer, and the 
position from which photographs are taken indicates whether the 
photographer has takeu the role of observer or participant. 

The presence of the photographer does not change anyone's 
behavior in the long run because observing «M photographing 
become part of_the scene. The reactive effect may be of 
surprisingly short duratfion^ In many situations, people being 
observed are engaged in 'ordinary activities that are important to 
them. They could no*; change what they are doing f o - an observer 
even if they waftted to. 
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It is important to oear in mind that reactivity itself often 
produces a significant form of data. How people react becomes 
one part of the photographer's exploration of the program, and 
facilitates the evaluator's relationship with participants. The - 
reaction of participants to the intrusion of an observer tells us 
much about how the participants feel about the situation in which 
they are involved. Different types of reactions to an observer 
or evaluator may indicate that the subject is under stress or is 
comf or table *with the situation, for example. In the case of 
Photographic observation, the very action that produces 
reactivity— taking pictures— also records information about 
reaction and preserves it for later analysis. The photographs 
become a record of the photographer's relationships with the 
people he or she photograpns (for example, in Owens, 1973 j 
Banish, 1976; Meiselas, 1975). 

The actual act of photographing need not be highly 
obtrusive, with modern films and equipment there is no need to 
use flash under the lighting conditions found in most offices and 
classrooms. If there is much activity going on, the noise of the 
camera will usually not be very noticeable and photographers, 
like any other type of observer, can learn to move about in the 
situation without calling undue attention to their presence. (In 
fact, the skill of entering a social situation gracefully without 
disrupting it and to remain unobtrusive is a skill that most 
documentary photographers must develop in order to carry out 
their work successfully— just as an evaluation observer must do.) 

Reactivity often reflects the program participants' 
apprehension and fear aoout what will be done with the 
information being gathered and photographed. The observer must 
give evidence that the participants will not be put in^eopardy. 
In some situations administrators may .want participants to sign 
release forms, and participants may prefer to sign release forms 
establishing the photographer's right to be there. Standard 
release forms are available in photography stores. 
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Collecting the Data 



Assemble photos, 
narrative materials 
for report 



Rephotograph, 
repeat process 



Validate, confirm 
attempt to 

disconf irm story, issues 

Go back, photointerview, 
interview, observer, 
rephotograph, gather verbatim 
data, repeat 

Select representative 
pictures, focus 
issues, topics 

Photograph mapping 
shots, detail and 
close-up shots 



Talk with 
client 8, program 
staff, 
• audiences 



Select sampling method 

Select instances 
to photograph 

Develop 
program 
issues 

Select 
photographer ( s ) 



Discover concerns 
of constituents, 
audiences, infor- 
mation needed 

Obtain 

permissions to 
photograph 

Negotiate access 
with camera 



Begin taking 
many pictures 



Adapted from stake (1*?5, 1980); Becker (1974, 1979); Guba and 
Lincoln (1981); Collier (1967, 1979). 



The data collection process is circular in nature, events 
occurring in differing sequences from one evaluation study to 
another. The photographer enters the process at any point in 
these prominent events on the handless clock, usually at 12 
o'clock* 
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Establishing Validity 



In evaluation with photography, validity has to do with two 
fundamental issues: establishing that the photographs are 
trustworthy evidence and that they reliably and accurately 
represent the program under study. Photographs convey a sense of 
truth, but they can misrepresent reality if they are distorted. 

Threats to Validity 

Photographic evidence can easily be distorted in several 
ways i by alteration of the photographs, by artistic intent of 
the photographer, by inadequate sampling of the program, and by 
censorship of the photographs (Becker, 1974, 1979), These four 
threats to the validity of evaluation with photography will be 
discussed hire, and then strategies will be offered for affirming 
the validity of an evaluative study. 

Altered photographs . The most obvious threat to the validity 
of inference based on photographic evidence is faking or 
doctoring the photographs in some way. Alteration includes 
retouching, cropping prints, using composite negatives, using 
models, arranging or staging the scene, and the like. These are 
ways of changing or excluding information so as to distort 
important content. 

Artistic intent . The photographer may distort images for 
artistic purposes. The desire to make art may lead photographers 
to suppress details that interfere with their artistic 
conception. This practice is acceptable in other contexts, but 
it makes the photographs unsuitable for use as evidence in 
evaluation. Some of these distortions may be readily detected by 
examining the photographs. For example, high or low angles of 
view can give a distorted impression of the relationship between 
people, between people and the environment, or can misrepresent 
the people themselves. The use of a wide-angle lens may give the 
impression that objects and persons are thinly scattered in the 
environment, whereas a telephoto lens tends to produce the 
opposite effect, that of bunching subject* together. Dim 
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lighting and blurred images tend to obscure information. There 
are other ways besides these of making artistic photographs; in 
evaluation, natural, uncontrived photographs provide clearer 
information. 

Inadequate sampling . Inadequate sampling can bias the 
picture of reality that emerges from the photographs, without a 
sampling plan, a photograpner could take many pictures but still 
not obtain a representative sample of the people and events in 
the program being evaluated. Without a plan, the photographs may 
not capture adequate representations of people's faces, spatial 
relationships, and social interactions. Furthermore, evidence 
about the physical settings in which programs take place can be 
easily distorted if the settings are not amply photographed. 

An important cause of inadequate sampling is limited access 
to the program. Without full access to the setting and people, 
the photographer may not be able to take pictures of the full 
range of activities in the program. Another cause of inadequate 
sampling is the failure to raise the issues in the program or to 
use some telling theory to guide the interpretation of 
photographs from day tc day. which images have meaning is 
determined in part by the program issues that are being raised 
during the evaluation. Without program issues, the photographer 
may tend to take pictures only of What is of personal interest to 
the photographer. 

Censorship > Censorship limits the pictures the photographer 
can take. Censorship may be imposed in a number of ways, but in 
evaluation the greatest threat censorship can make to validity is 
the compromises the photographer may have made in negotiating 
access to the program— for example, denial to photograph people's 
faces, or have access to certain meetings, events, and 
records — anything someone may not "ant known. Such aspects might 
be vital to a full appraisal of the program. If some form of 
censorship limits the photographer's access to any part of the 
program, then the audience. of the evaluation will be prevented 
from obtaining certain information about the program. Whatever 
has been withheld may be pertinent to representing some 
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perspective of the participants that might substantially change 
the interpretations that can be made of the photographic evidence* 

Strategies to AS* inn Validity 

Establishing the validity of photographic data is 
accomplished in much the same way as for other types of data—by 
specifying the conditions of data collection and adhering to the 
procedures for reducing threats to validity* Specifying these 
conditions in the final report should be a standard part of any 
photo-assisted evaluation. 

There are several important strategies for affirming the 
validity of photographic evidence* By using these strategies, 
evaluators attempt to achieve fair representation of the program 
by overcoming the major threats to validity that were outlined 
above. The strategies for establishing the validity of 
photographs in evaluation include: 

Cross checking photographs with participants 

Representing multiple perspectives 

Following a sampling plan 

Establishing continuity of time 

Keeping complete records 

Leaving photographs unaltered 

Gaining full access to the program 

Staying on-site as long as possible 

Using multiple methods 

Each of these strategies will be described briefly here* 

Cross checking photographs with participants * Cross checking 
evidence is the most important method of establishing the 
veracity of photographic evidence* It should occur periodically 
and continue throughout an evaluation study. The basic notion of 
cross checking is that the program's constituent audiences have 
the right to provide feedback to verify, modify, or t reject the 
evaluator's photographs and other findings* 
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Cross checking is similar to the concept of tnangulation in 
navigation— getting a fix from three different physical points. 
In educational evaluation, cross checking the photographic and 
narrative evidence with at least three sources among the 
constituents—for example, the teachers, students, parents, 
administrators, specialists, etc. —is another form of 
triangulation that provides a reality test. Evaluators and 
photographers have an obligation to sift and classify information 
gathered about the program's issues and apply their analytic 
resources in interpreting the information, and then take it back 
to the informants for cross checking. The result may be a 
negotiation process from which more clear representations of 
people's differing perspectives about the program emerge. 

The report should include a number of prints taken in the 
study. It should also include proof sheets for independent 
checkir . 

Represent ing multiple perspectives . A key strategy in a 
photographic evaluation involves the question of whose 
perspectives on the program is being represented. Each group of 
constituents may represent varying points of view about a 
program. In the interest of thick descriptions and fair 
representation, the photographer and evaluator should present the 
differing views. Without a range of views, the evaluation may be 
biased to the extent that both description and interpretation are 
limited. A more full description represents the views of 
different people even if they are opposing views. The process of 
gathering these multiple perspectives is compatible with the 
process of cross checking the issues. Both photographic and 
narrative reports of information about program issues need to 
represent the many perspectives in fair proportion. For example, 
in Serrano's Western City evaluation (1978), a letter of 
disclaimer was included to represent an administrator's 
perspective that opposed Serrano's perspectives represented in 
photos of racially segregated students. 

Following a sampling plan . At the outset the sampling plan 
selected is a method of directing the photographer to represent 



34 



events and participants in the program in a way that provides a 
fair picture of what is transpiring. The intent is to assure 
that sampling and data collection have been suf f iciently 
extensive so as not to be biased on the side of representing only 
the photographer's and evaluator's perspective. 

Establishing continuity of time . This refers to methods for 
accounting for how much time was spenc on-site and the order of 
sequences in photographs. There is nothing inherent in photos to 
indicate time, but va infer a sense of time indirectly. When we 
view a sequence of pictures, we assume that they were taken in 
the order in which they appear. The longer the photographer is 
on-site the more we tend to trust the photographs. For example, 
Owens spent two years on * the site taking photographs for 
Suburbia. By keeping an accurate log, records of film exposed, 
and presenting proof sheets, the photographer can disclose 
continuity over time. This is important for making inferences 
and comparisons of photos. The major result is to assure the 
reader that the coherence implied in a sequence of photos is not 
contrived by biased selection, but rather accurately represents 
events that occurred in order over time. 

Keeping complete records . This refers to providing a 
reasonable if mundane way of establishing the credibility of 
photographs and reducing doubts. Keeping a log of film exposed 
provides a basic record of the organization of the photographic 
work— dates and location where film was shot, number of rolls 
exposed, prints made and selected, subjects photographed and 
interviewed. By disclosing record-keeping procedures, the reader 
should be able to determine that all types of data have been 
reliably collated and, hence, that errors in interpretation have 
not arisen from faulty record keeping. The reader should have 
adequate indications of the number of exposures made and the time 
period covered, and should have access to proof sheets or raw 
data if possible* The report should also make clear any gaps in 
the photographic coverage. 

Leaving photographs unaltered . Photographers must decide on 
ways in which they are going to present photographs, whether by 
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whole frames, keeping track of sequences, and eschewing technical 
altering such as cropping. They need rules for handling sampling 
and rules for data. Altering photographs used for evaluation 
purposes would not appear to be much of a problem because of the 
rigor of methods applied. Evalu&iors can reduce error and bias 
and increase the informational value of photographs by not 
altering or distorting them. Evaluators should disallow and 
avoid using cropped photos, faked subjects, posing subjects, 
staging props and physical arrangements, out-of -focus prints, 
extreme lighting, selecting distortions of subjects, and any 
other form of altering what is pictured, omitting legitimate 
evidence, or compromising information in the prints. 

Gaining full access to the program . Negotiating access to 
the program is of key importance. Agreements about what people 
and settings will be available or restricted to the photographer 
determine much about the quality of evidence. Having 
restrictions implies that certain information is going to be 
compromised or not available. Excluding, inhibiting, and 
censoring certain kinds of information can seriously bias the 
evaluation and the kinds of inferences that can be made about 
photographs and interview data. 

The evaluate* should be able to photograph and interview as 
wide a variety as possible of constituents in the program, 
including students, teachers, parents, specialists, 
administrators, consultants, other evaluators, and taxpayers in 
general. Having obtained permission to move freely about the 
site, the photographer should also have unrestricted access to 
all of the program's environment. 

Staying o n-site as long as possible . The longer the 
photographer has access to events, people, and spaces to 
photograph, the greater the power of description and the 
credibility of the photographer, and the more likely the 
appraisal can inform decision makers about what occurs in the 
program. Staying longer increases the accuracy and stability of 
interpretations and assertions made about photos. But time is 
often short in. evaluations because of budget restrictions and 
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design requirements. Documentary style of photography is more 
credible the longer the photographer has time to make photographs 
and interview participants. Nevertheless, when a longer time is 
not possible, observing other strategies for establishing 
validity can improve the quality of evidence. £^3%* 

Using mul tiple methods . This refers to gathering as many 
kinds of relevant data as possible from various sources in the 
program, using a combination of methods. Photographs alone are 
not sufficient fro most evaluations. 

An unusual example of a photographic evaluation using 35cun 
color slide photographs only is Bernadine Stake's 1973 evaluation 
of a children 1 s arts program. The evaluation audience required 
that pictures be taken of the teaching methods and processes 
children were learning for handling eight major art media. The 
lack of multiple methods was made up by three other factors: the 
long time she spent on-site (an academic year), the large" number 
of slides taken (1600), and the weekly reporting meetings with 
program staff. Each week the staff viewed all slides taken the 
previous week to interpret for themselves the evaluation results. 

In addition to photographs, other methods to employ generally 
are obeerving, interviewing, document analysis, and surveying. 
Taking narrative notes, interviewing, audio recording, and 
photo-interviewing are essential methods of describing a range of 
program elements. Photographs and scenarios can be used as 
projective instruments. Demographic and survey data will be 
helpful in fleshing out the photographic data. Pertinent 
historical records, project proposals, written materials that 
pertain to the subject matter, curriculum guides, schedules and 
the like will all reveal important information to provide fuller 
program description than photographs alone. A variety of methods 
produces these materials. Final reports should reflect these 
multiple methods and their results. 
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Interpreting the Data 

This section suggests several methods evaluators might choose 
for interpreting groups of photographs, depending on which seems 
appropriate for a particular study. The main idea is to discover 
themes, patterns, and issues by grouping photographs that go 
together to provide the primary supporting evidence. 

The interpretation of photographic evidence suggested here 
implied that generalizability is a matter of logical inference 
that is plausible and validated against experience (Stake, 1980), 
rather than causal explanations validated with empirical proofs. 
The logic of inference depends in part on the Jcinda of questions 
asked. The questions emanate from ideas about what the 
investigator anticipates finding, is finding, and is analyzing 
about in the situation under study. 

How do we make sense out of so many photographs and find 
meaning in the ones that are selected? Program issues and 
telling theories (held lightly, to not confirm theories but to 
guid* inference making) help structure the study, and shape the 
procedures for sampling, collecting data, and interpreting the 
results. The program issues direct the photographer to whole 
classes of information in the photographs. The photographer 
looks for alternative interpretations to reformulate the 
interpretation, meanings that may emerge suggested by the data 
about what is happening. ^\ 

The basic idea is to clarify how the photographer thinks 
things really are, and to present photographs that use imagery as 
evidence for inferences and assertions that have developed 
(Becker, 1979). Broudy (1979) contended that images are the 
first news of knowledge and the basis of concept formation. The 
interpretation made is a function of the issues as they are found 
and cross checked in photographic and narrative data, &nd the 
kinds of questions asked of the data. 

McCutcheon (1981) described three types of interpretion in 
qualitative inquiry that seem to be useful ways of thinking about 
photographs* They include looking for patterns that form; sociffl. 
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meanings as interpreted by insiders (program participants) and 
outsiders (the photographer/evaluator ) ; and relating observed 
internal events to various external factors. 

As a way of inferring social statements about photographs, 
Becker (1974, 1979) raised a list of sociological questions that 
can be answered photographically in the field. He cautioned that 
answers do not come all at once, but through a process of 
progressive refinement and constant testing against new 
information that contains visual counterparts, 

1. What are the cast of characters and status groups? 

2. What expec tat ions does eac^ kind of person have for 
norms , rules, or common understandings? 

3. What are the typical breaches, deviance to rules? 

4. What sanctions or conflict resolutions occur when 
expectations are violated? (Becker, 1974) 

Seeing an instance of one of these concepts alerts the 
photographer to look for other^visual counterparts of the 
interpretation* 

Collier (1967, 1979) contended that there are twb levels of # 
classifying involved in the inferential process* T$e first level 
of classifying is a descriptive account that counts, measures, 
and tradrs materials, elements, and cultural/social 
arrangements. The second level of classifying is an interpretive 
account, identifying and evaluating the overall patterns of the 
subgroup culture or situation under study, based on available 
data. This* correlation is made not by counting or measuring but 
by the photographer's making judgments about the visual evidence 
of personality, behaviors, and the character of the situations. 
Developing, disconf irming, and confirming the validity of such 
judgments is accomplished through photointerviewing subjects, 
using the provocative quality of pictures as the interpretive key 
to situations? and then through the photographer's calling on his 
or her own judgments to clarify how the photographer thinks 
things really are, to express inferences and assertions that have 
been informed and stimulated by the multiple evidence t 4 at has 
been gathered. 
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Because we must select from the great number of photos taken, 
the interpretative task is basically a procedure of 
classification and sorting photographic evidence. The attempt to 
answer the kinds of questions cited here is interwoven with 
further photographing in a process of simultaneous data 
gathering, analyzing, and reconceptualizing that begins with the 
first selections of pictures as they pertain to program issues, 
and continues throughout the evaluation. Making valuations of 
worth, locating strengths and weaknesses in the program are not 
reserved for the final analysis only, but are embedded throughout 
the photographing/evaluating process. The point here is not to 
be pedantic or formalistic by neatly fitting pictures into 
categories, but to grasp meanings so as to quicken insight and 
inform understanding about the program. Selecting and grouping 
photographs around common themes and issues build the categories 
for which the photos are primary data, supported by narrative 
data. These categories also structure the report. 

Development of the interpretation is both a tentative and 
ongoing process during the on-site phase of the study. The final X 
part of the process draws together the various types of data 
collected and unites them with the overall system of description 
that has been evolving through progressive focusijjg of initial 
program issues. Gathering data, both narrative and photographic, 
and making judgments about the "description is close to Geertz'o 
definition of thick description.' description layered with 
interpretations and judgments. 

Prom the standpoint of how this applies in a report using 
many photographs, the key is that the logical inferences made 
from photographic evidence must be accessible to the reader, in 
more or less the same way that inferences from quantitative data 
are presented in a form that allows a reader to be assured of the 
strength of the inference. In the case of photo-assisted 
evaluation, this task may appear as a clear, narrative 
explanation of the story, or as lengthy summaries of issues from 
observations or interviews with participants, or as statements 
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letting participants speak for * es ( that take on the form 

of title ard captions for tha one ,* * 

Reporting the Evaluation Uaincy Photographs 

The transformation of photographic data into project results 
is often leas clear and dramatic than the transformation that 
quantitative Oat* undergoes, A photograph appearing in the final 
report is, after all, physically the same as a photograph in the 
mass of project data. The difference is that the selection of 
photographs for inclusion in the report is performed as a means 
of achieving economy of representation, while still providing 
examples of <Jata that will export the inferences made in the 
report. 

Project reports based wholly or partially on photographic 
evidence must often ditfmr physically from traditional reports. 
Ordinarily, project reports are simply the final, typewritten 
copy of the results, reproduced in as large aoantity as 
•necessary by whatever means available (mimeograph, xerography, 
^irit copying, offset printing, even carbon paper), the 
inclusion of photographs immediately limits the proc sees that 
can be used, and in any case multiplies the cost of producing a 
report. The standard format of reports (letter size, or 
8- 1/2 X 11 inches) is also less than ideal j Larger pages would 
msXe it easier to present groins of photographs together (for 
comparison or sequence), with the nerrative interpretation 1 
material kept close by, for the reader's convenience. However, 
any non-standard format will multiply production costs, drive 
librarians to distraction, and perhaps reduce readership because 
the report looks unwieldy or unprofessional, 

Cheatwood (1979) identified three ways of presenting 
photographs: the slide presentation, the book, and the mounted 
show or gallery display. The book format is organised into 

traditional essay, paper, report, or book in which photographs * 
are mounted, pasted, -printed, xe xed, or overlaid on pages which 
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include to greater or lesser degree some narrative text. That 
is, numbered photographs are arranged, ordered, and grouped in 
meaningful ways that tell a story. They are accompanied by text 
and titles* The best negatives for this format come from cameras 
that produce 35ram and 2-1/4 inch negatives. 

For some purpoiies, of coarse, a hard copy rei>ort is 
unnecessary, in whiah-case the photographic data may need to be 
incorporated into a presentation. Black-and-white negatives can 
be converted readily to black-and-white slides by contact 
printing the strips of negatives onto strips of what is known as 
positive print film. The process is fast and inexpensive and 
results In slides that can be mounted for projection in a 
standard 35mm slide projector. Properly sequenced and 
identified, the slides can then used both to illustrate and 
support the interpretations in a presentation of the project 
results. 

It would also be possible to combine aspects of the written 
report and the slide show presentation. The test of the report 
could be presented in the normal, printed fashion, accompanied by 
a Carousel or other magazine of properly sequenced slides. The 
written text would contain indications of slide msabers for the 
reader to project and examine while reading, if only a few 
copies of the report were to be produced, this format might 
actually be less expenisve than proviiing a set of original 
photographic prints in each copy. Some readers may also find it 
easier to examine the large, projected image than to examine a 
print, e^>ecially if the prints have been kept to modest size to 
save space and printing costs. 

Whatever format is chosen for the final report or 
presentation, certain criteria need to be met in order to achieve 
clarity: 

1) Photographs and the narrative relating to them must 
be kept in very close proximity to each other, 
preferably on the same page or facing pages. This is 
standard practice in book design, but it must be 
observed very closely in photo-assisted studies, 
because the reader must be able to grasp photographic 
and verbal interpretation together. 
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2) Photographs must be reproduced with sufficient size 
and sharpness so that the inferences made in the text 
can be verified by the reader by examining the 
photographs. If original prints are included in the 
report, this will usually not be a problem, so long 
as the prints are not miniscule. However, if the 
photos are printed as halftones along with the text, 
the photographs will usually have to be larger, and 
it will be necessary to have the printing done on a 
good grade of paper, using a fine halftone screen 
(100 lines per inch or finer). 

3) A figure numbering system must be used so that the 
photographs can be referred to in the text. If the 
report also includes extracts from field notes and 
photo-interviewing sessions, than the photographs may 
also have to be identified by roll and frame njmber, 
or whatever system was used to identify them during 
the course of note- taking and interviewing. 

4) Narrative text and photographic data may not be 
themselves be sufficient content for a report. In 
order to establish the validity of the photographic 
evidence and complete the description of he research 
methodology, it may be necessary to include 
additional documentation. Complete contact sheets, 
tables correlating film exposed by subject and date, 
sampling plans, photographing protocols, and 
photo-interviewing protocols may need to be included 
or described in summary form in order for the reader 
to validate and accept the interpretations of the 
photographic evidence. 

The final results of photo-assisted evaluations are quite 
likely to be bulkier than reports generated from other types of 
data, for two reasons. First, photographs as results do not 
condense data in the same way that tables, graphs, and statistics 
can be said to condense their quantitative antecedents, A 
photograph, however used or in whatever context, remains an 
extended, nondiscursive representation, in an analog rather than 
a digital moSe, a point from which verbal inferences and 
descriptions expand rather than toward which numerical contract ♦ 
Thus, although a picture can be said to be "worth a thousand 
words* for descriptive purposes, it may also serve to generate 
thousands of words as we tease out inferences from pictorial 
information and struggl, r? convert nondiscursive symbols into 
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another system of rr?resentation--language--vhdch is also 
symbolic, but discursive in nature. 

And this is the second reason that photo-assisted evaluation 
may appear uneconomical in its results, in the veibal 
interpretation of visual information, We are still in our 
infancy, m most stances we have no shorthand notation to render 
our inferences and descriptions verbally compact. Two notable 
exceptions are notations for kinesics (Birdwhistell, 1970) , and 
dance notation. For the most part, our inferential/descriptive 
language is a mixure (and not always systematic), borrowing from 
the vocabularies or art history, a^thetics, literary criticism, 
philosophy, and, at * mes, the figurative language of the poetic 
tradition. Prom one aspect, it /is a bulky system of 
representation, but from another it may be seen as a means of 
bringing us closer to the realization of what staxe (1975a, 
1975b) called responsive evaluation—evaluation that provides 
needed information for all ?oncerned by portraying interactions 
rather than outcomes, and by representing multiple perspectives 
from within the ac.-v*. In the long run, an evaluation 
methodology incorporating photography may be one of the richest 
and most sensitive means of achieving this goal of responsive 
evaluation. 



Summary 

This handbook has described important elements of a method of 
photographic inquiry for evaluation, what it tells evaluators is 
that to incorporate photographic methods is of minimum threat, 
and does not require wholesale revision of the process of 
evaluation* Rather, to add photography would require only an 
expansion of existing evaluation methods with photography. 

More attention appears to have been given to problems of 
various threats to validity and sampling plans. This does not 
mean that other elements of method are nbt important 
considerations. The attention given nere reflects the attention 
given to these matters in the literature of photographic 
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inquiry. Other methodologic elements need to be developed by 
evaluators as they adapt photographic methods. For example, the 
problem of history is of prominent concern, especially since I 
advocate staying on the site as long as possiDle to collect data 
and interpret it. The fact that we can discuss such elements in 
evaluating with photography attests to the possibilities for its 
congeniality with other evaluation methods. 

The emphasis in this handbook is on the aspects of 
educational evaluation dealing with questions about what 
transpires in the program. How useful photography is to 
evaluators may be viewed as a function of how important they 
think descriptions and interpretations of program operations and 
activities are rather than learning outcomes. The descriptions 
of events, activities, and social interactions in a program 
represent pictures of the social organization of the curriculum 
and learning conditions. Using direct observation with a camera 
in natural settings, Goffman (1980) argues th-t we can photograph 
social interactions but not organizations. He contends that it 
is in small social interactions that people organize perceptions 
and derive meanings from their experiences. Translated to 
educational evaluation, such a view can provide a method for 
interpretive appraisal of the program's interactions. 
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